Efficient Federated Learning for AIoT Applications Using Knowledge Distillation

As a promising distributed machine learning paradigm, federated (FL) trains central model with decentralized data without compromising user privacy, which makes it widely used by Artificial Intelligence Internet of Things (AIoT) applications. However, the traditional FL suffers from inaccuracy, since local models only using hard labels while useful information incorrect predictions small probabilities is ignored. Although various solutions try to tackle bottleneck FL, most them introduce significant communication overhead, making deployment large-scale AIoT devices great challenge. To address above problem, this article presents novel distillation-based (DFL) method that enables efficient and accurate for By knowledge distillation (KD), in each round training, our approach uploads both soft targets gradients cloud server aggregation, where aggregation results are then dispatched next training. During DFL addition labels, approximate targets, can improve accuracy leveraging targets. further performance, we design dynamic adjustment strategy loss function weights tuning ratio KD maximize synergy between labels. Comprehensive experimental on well-known benchmarks show significantly introducing overhead.